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What constitutes effective feedback to postgraduate research students? 
The students’ perspective 


Abstract 

Many Western universities are experiencing considerable growth in the numbers of postgraduate research 
students, both local and international. This increase and diversification bring with them challenges for how to 
make these students’ research studies successful. In particular, what students may wish to receive byway of 
supervisor-student relationships, and feedback within those relationships, may differ from what supervisors 
give, thereby creating potential tensions in the relationship and hindering effective learning. This article looks 
at what research students report they receive by way of feedback from supervisors, and what they say they find 
most effective. Evidence from questionnaires (n = 53) and interviews (n = 22) is used to draw some 
conclusions about how effective feedback is conceptualised from the students’ perspective. Analysis includes 
similarities and differences in response for students who speak English as a first or additional language. 
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Introduction 

Postgraduate research supervision is an important component of the university teaching and 
learning environment. Although it differs fundamentally from the more didactic, less "personal" 
and more "short-term" teaching approach often adopted in larger-scale undergraduate and 
postgraduate taught courses, at the heart of supervision is pedagogy. As Wisker et al. (2003, p387) 
put it, postgraduate supervision “should be seen as a form of teaching,” and postgraduate research 
“as a form of learning.” Fundamental to this teaching and learning is supervisor feedback offered 
in the broader context of an ongoing relationship, and what Wisker et al. refer to as "learning 
conversations", the ultimate goal of which is to bring the learner to a place of independence from 
the teacher. 

As with any teaching/learning process, the supervisory process is not unproblematic. Grant (2005, 
p2), for example, speculates that “[t]he single word 'supervision' obscures a great and sometimes 
troublesome diversity in values, beliefs, assumptions and practices”. What the student wants to 
receive by way of feedback may sometimes differ from what the supervisor gives, thereby creating 
potential tensions in the supervisor-student relationship, and marring its effectiveness. 
Nevertheless, as Pearson and Kayrooz (2004) argue, effective supervision is critical to students’ 
satisfaction with their postgraduate experience and successful degree completion, even though a 
robust conceptual understanding of what supervision involves is often lacking. 

Given the increasing numbers of students, both local and international, who wish to undertake 
postgraduate research studies (as noted, for example, by Wisker et al. (2003) and Pearson and 
Kayrooz (2004)), and the "diversity in values, beliefs, assumptions and practices" that this diverse 
student body will bring into the teaching and learning context (as noted by Grant (2005)), 
postgraduate supervision practices must arguably become a greater priority for research. Wisker et 
al. observe, however, that research into “approaches to learning, [and] preconceptions and 
misconceptions about learning required in different subject areas” has “largely focused on 
interactions at undergraduate level” (p383). Johnson et al. (2000, p 135) similarly propose that 
learning and teaching at the postgraduate research level - in other words, postgraduate supervision 
- “has largely remained unscrutinised and unquestioned.” 

This article reports findings from an exploratory study to investigate what supervisors and research 
students in New Zealand universities identified as effective feedback practices. The primary aims 
of the study were to identify similarities and differences in supervisor and student viewpoints 
within and across contrasting disciplines, with a particular focus on written feedback. The broader 
study drew principally on questionnaires and interviews with supervisors and students (excluding 
supervisor-student pairs) drawn from three discipline areas (humanities, commerce and 
sciences/mathematics) in six New Zealand universities. We have reported elsewhere on "effective 
feedback" as perceived by supervisors (see, e.g., Bitchener, Basturkmen & East 2010). In this 
article we focus on the students’ perspective, in particular the feedback on written work that 
students report receiving, and how this feedback works for them in the context of the "learning 
conversations" (both written and oral) that take place in the supervisor-student teaching-learning 
relationship. The article also discusses findings with regard to students who spoke English as 
either their first language (LI) or an additional language (L2). Our findings provide empirical 
evidence of what has thus far often only been anecdotally acknowledged about how postgraduate 
students construct the supervisor-student relationship and the effectiveness of the feedback 
embedded within it. 
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Background 

Feedback in the context of a pedagogical relationship 

In contrast to the short-term (i.e., often single-semester), monodirectional (i.e., lecture-focused) 
and didactic (i.e., teacher-led) teaching approach common to many undergraduate, and in some 
cases postgraduate, courses, postgraduate research supervision requires “a symbiotic and cordial 
relationship” which is “collaboratively developed and sustained” (Li & Seale 2007, p511). This 
relationship is essentially pedagogical in nature. Franke and Arvidsson (2011) argue that research 
supervision involves both a knowledge process and a relational process through which the 
research student is given the opportunity to develop the knowledge and skills needed to carry out 
research effectively. As a consequence, “[a] well-functioning pedagogical relationship ” between 
supervisor and research student is “emphasised as being an important factor in successful 
postgraduate education” (p8, our emphasis). The ultimate goal is to allow the student to operate 
independently of the supervisor. That is, the mark of students’ success will be “the extent that they 
become independent lifelong learners who have learned from us but no longer depend on us to 
learn” (Riordan & Loacker 2009, pi81). 

A crucial element to "becoming independent" is effective feedback (Lantolf 2000). Hattie and 
Timperley (2007) maintain that feedback exerts significant influence on learning and achievement, 
and has considerable power to improve teaching and learning. Effective commentary on students’ 
work is a key characteristic of quality teaching (Ramsden 2003), and supervisors’ constructive and 
detailed feedback on written work has been identified as a key characteristic of good research 
supervision (Engebretson et al. 2008). Feedback on writing plays a crucial role in the enculturation 
of students into discipline-relevant literacy and epistemologies (Hyland 2009). Kumar and Stracke 
(2007, p462) argue that “it is through written feedback that the supervisor communicates and 
provides advanced academic training, particularly in writing, to the supervisee.” The central 
importance of feedback for student writers is therefore well established in the literature (Benesch 
2000; Hyland & Tse 2004). 

In essence, then, feedback is embedded within a "symbiotic" relationship (which must therefore 
function successfully in the medium to longer term), and feedback is crucial to the development of 
research students’ ultimate independence (and must therefore be working to that goal). However, 
differences in understandings about what constitutes an "effective relationship" and "effective 
feedback" may present significant challenges for both supervisors and students. These differences 
will potentially put a strain on the supervisor-student relationship and will hinder effective 
learning. 

What constitutes an "effective relationship"? 

The pedagogical relationship at the heart of postgraduate supervision has been conceptualised in a 
variety of ways: Grant (2008, p9) draws on the analogy of "master-slave" as a “significant 
archetype for supervision.” Other analogies include the supervisor as "effective manager" 
(Vilkinas 1998); the student as "apprentice" (Li & Seale 2007); or supervisor-student in a 
"professional-client" relationship (Li & Seale 2007). Although, as Pearson and Kayrooz (2004, 
plOO) argue, “[t]erms such as 'apprenticeship', 'mentor' and 'coach' are used frequently in 
discussing supervision,” this discussion is often carried out “in ways that perpetuate the 
mystification of the process.” Wisker et al. (2003, p388) propose “a role continuum ranging 
through dictator/authority figure/‘God’, manager, guide, mentor, facilitator, collaborator, friend, 
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counsellor, mother/father.” This variety of conceptualisations reveals the complexities involved in 
defining the relationship that exists between supervisor and student, and, therefore, in determining 
what makes it "effective" for both parties. As Wisker et al. conclude, “[e]ach role has potential 
problems because of the long-term professional relationship between supervisor and student” 
(p388). In turn, each role has potential problems because students and supervisors may construct 
the relationship differently, and approach the relationship with different expectations. 

What constitutes "effective feedback"? 

Carless et al. (2011, p395) make it clear that, on the one hand, “[f]eedback is central to the 
development of student learning.” On the other, they suggest that “feedback is clearly an issue in 
need of further analysis.” Arguing in the context of undergraduate written feedback, they go on to 
suggest that this is because students may have difficulty understanding feedback comments, are 
unsure what to do with them or may find them terse and operating in a one-way (teacher to 
student) direction. Even at the postgraduate supervision level, where feedback is embedded within 
more bi-directional "learning conversations", supervisors and students may approach the concept 
of "effective feedback" from a range of perspectives. 

In the supervisor/student relationship, the beneficial power of feedback can be significantly 
influenced by the different expectations of both parties. The goal might be clear: that is, “[t]hrough 
discussion and feedback, the supervisor requires the student to think again, to read more, to revise 
written drafts” (Grant 2005, p76). However, the message might not always be clearly received: “at 
times these requests are likely to be delivered confusingly or inadequately by supervisors or 
misunderstood, resented and resisted by students (much more so by some for reasons of politics, 
biography, or temperament)” (p76). 

Li and Seale (2007, p514) observe that clear communication can be obstructed “when there are 
cultural and linguistic differences between students and supervisors.” For example, students from 
a Confucian Heritage Culture may tend to adopt a learning approach that involves deference to the 
supervisor as the one "in authority", and focusing on factual knowledge in a relatively uncritical 
way, standpoints that are “somewhat at odds with the problem solving dialogue with experts mode 
of European and western research” (Wisker et al. 2003, p384). As Aspland and O’Donoghue 
(1994) note, language barriers, and lack of culturally specific knowledge about the demands of 
postgraduate research, can hamper effective communication between international students and 
supervisors. In turn, poor communication can contribute to the collapse of the relationship 
(Delamont et al. 2000). 
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Effective feedback from the students’ perspective 

If, therefore, the supervisor-student relationship is to be maintained, and feedback is to be 
effective, account must be taken of the different understandings, cultural backgrounds and ways of 
learning that students might bring to the relationship. Li and Seale (2007) note, however, that 
where differences exist in the relationship, there is a tendency to favour the supervisor’s language 
and culture. It is therefore important to consider the students’ perspective to determine what they 
say works for them. 

One wide-scale study into feedback from the students’ perspective (Gulfidan 2009) used 
questionnaires and interviews to investigate social-sciences doctoral students’ perceived needs and 
attitudes to written feedback. This study provided a 12-fold categorisation of aspects of writing on 
which students perceived that they needed feedback (p71). In summary, Gulfidan found that the 
students expressed a preference for "straightforward written feedback", "feedback that gives me 
clear instructions for how to revise my paper" and "detailed, specific comments more than overall, 
general comments". Amongst negative statements for preferences were "suggestions that are hard 
to use while revising my paper", "marks without text (such as, underlined sentences and question 
marks)" and feedback that "tries to change my writing style" (pp74-75). 

The student perspectives identified by Gulfidan (2009) support Hattie and Timperley’s (2007) 
arguments that “[feedback needs to provide information specifically relating to the task or process 
of learning that fills a gap between what is understood and what is aimed to be understood” (p82) 
and that “[sjpecific goals are more effective than general or nonspecific ones” (p87). The study on 
which this article focuses sought to add to our understanding of what constitutes effective 
supervision of postgraduate research students, with particular emphasis on written feedback, and 
thereby to shed more light on what is and is not working. Although, as noted at the start of this 
article, the phenomenon was investigated from both supervisor and student perspectives, this 
article focuses on the students’ perspective and presents data generated from questionnaires (n = 
53) and interviews (n = 22). It addresses four research questions: 

1. What types of written feedback do students report that they receive? 

2. What types of written feedback do students report finding the most helpful? 

3. What differences, if any, exist between students with English as LI and English as L2? 

4. Based on their experiences with feedback, what recommendations would students make to 
new supervisors? 

Design 

The overarching goal of the study was to examine effective feedback from a range of angles for 
purposes of triangulation, complementarity and expansion (Greene & Curucelli 1997; Greene, 
Curucelli & Graham 1989; Tashakkori & Teddlie 2003). Participants (students and supervisors) 
were drawn from three discipline areas in six New Zealand universities (humanities, 
sciences/mathematics and commerce). It was not the purpose of the research to investigate 
supervisor-student pairs. One requirement was that the research degree represented students’ first 
venture into postgraduate research, whether at the master's or doctoral level. 
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We drew on a mixed-method component design (Greene & Curucelli 1997) whereby different 
instruments were used as discrete aspects of the overall inquiry. Supervisors (n = 35) and students 
(n = 53) completed complementary questionnaires, and subsets (n = 22 and 22 respectively) were 
subsequently interviewed, using questionnaire responses as a basis for interviewing. The 
interviews were recorded. A complementary vantage point was provided by examining feedback 
given on samples of draft texts provided by students from each of the three discipline areas (n = 3 
x 5). 

A small-scale pilot study involving exploratory interviews with a small number of supervisors and 
students in two universities provided the basis from which questionnaires for the main study were 
generated. Interviewed students were also asked to bring to their interview a sample of written 
feedback they had recently received. In cases where feedback was provided, it was discussed with 
the student as part of the interview. 

Data analysis 

Data from the questionnaires were principally used to answer Research Question 1 (what types of 
written feedback do students report that they receive?). The focus of analysis was on frequencies 
with which a range of feedback practices were reported. Interviews with participants provided 
scope to explore students’ perceptions of effective feedback practices. Two out of every three 
interview participants had English as L2, and data from the interviews were used to answer 
Research Questions 2 and 3 (what types of written feedback do students report finding the most 
helpful? What differences, if any, exist between students with English as LI and English as L2?). 
The questionnaires and interviews each provided participants with the opportunity, based on their 
experiences with feedback, to make suggestions about the types of feedback practices they would 
recommend to those new to supervision. Data derived from both questionnaires and interviews 
were therefore used to answer Research Question 4 (based on their experiences with feedback, 
what recommendations would students make to new supervisors?). 

Findings and discussion 

The feedback students report that they receive 

In the questionnaires participants (n = 53) were asked to report on the feedback they received, 
considered across three broad areas: 

1. content - the content and subject matter of their work; 

2. organisation - the organisation and structure of their writing; and 

3. language - the accuracy of their language. 

Questions were open-ended to give participants the opportunity to report what they considered to 
be relevant examples of the types of feedback they received in each of the three areas under 
consideration. Responses were subsequently entered into a spreadsheet and categorised with a 
view to generating an essentially descriptive account of current feedback practices, as perceived by 
the recipients, and identifying the frequencies with which particular responses were given. Table 1 
gives the frequencies of response in each of the three broad areas noted above. 
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TABLE 1: Reported Types of Written Feedback on Postgraduate Thesis Drafts 


Type of written feedback reported 

Total/53 

Percentage 
of responses 

1. Content 

1. Relevance of literature 

14 

26% 

2. Rethinking how the work is being presented 

12 

23% 

3. Help with finding literature 

10 

19% 

4. Appropriateness of methodology 

8 

15% 

5. Other/no comment 

9 

17% 

2. Organisation 
and structure 

1. Specific comments on organisation 

25 

47% 

2. Overview comments on organisation 

16 

30% 

3. Cutting out or condensing irrelevant material 

7 

13% 

4. Other/no comment 

5 

10% 

3. Accuracy of 

language 

1. Vocabulary and appropriate register 

23 

43% 

2. Grammar, spelling and punctuation 

21 

40% 

3. Other/no comment 

9 

17% 


Subsequent % goodness-of-fit analyses were used to determine whether the frequencies of 
response in each of the three categories differed significantly from those that might have been 
anticipated if all responses in each category were noted equally. With regard to reported feedback 
on content, it was found that frequencies of response did not differ from what might have been 
expected, with % 2 (4, n = 53) = 2.19 and p = .701. That is, no categories of feedback on content 
stood out as being notably more or less frequent than any other. 

Significant differences were found, however, with regard to reported feedback on organisation 
and structure , with / 2 (3, n = 53) = 19.08 and p < .001, and on accuracy , with % 2 (2, n = 53) = 6.49 
and p = .039. Feedback on organisational structure appeared to focus predominantly on specifics 
of the organisation, with just under half of the participants identifying this dimension of feedback. 
Specific comments contributed the most to the 2 (at 10.4 of 19.08), indicating a tendency for 
supervisors to provide direct comments considerably more than global feedback on organisational 
matters. Feedback on language appeared (unsurprisingly, given how linguistic accuracy is 
generally perceived) to consist of comments relating to language choice and grammatical 
accuracy, and these comments appear to have been given in roughly equal measure. Reports on 
accuracy comments that were not related to choice of vocabulary or use of grammar contributed 
the most to the % 2 (at 4.25 of 6.49), indicating that these reported comments were in the clear 
minority. Such reports may have been attributable to some participants misunderstanding the 
intent of the question (for example, one participant noted, as an example of feedback on accuracy, 
a comment to "simplify the writing style", a comment that relates only indirectly to accuracy of 
language). 

Participants were also asked in the questionnaires to identify the "feedback system" their 
supervisors used. Three broad types of feedback system were identified (Table 2). 
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TABLE 2: Identified Feedback System for Postgraduate Research Students 


Feedback system 

Total/53 

Percentage 
of responses 

1 . Written feedback followed by face-to-face meeting 

40 

75% 

2. Only/largely oral feedback 

5 

9% 

3. Only/largely written feedback 

4 

8% 

4. Other/no response 

4 

8% 


Again unsurprisingly, given that by far the most common feedback system identified was written 
feedback followed by a face-to-face meeting, noted by three out of four participants, significant 
differences were found with regard to the identified feedback system, with x 2 (3, n = 53) = 72.06 
and p < .001. 

Conclusions drawn from these % 2 analyses should, however, be treated with caution. They are not 
based on frequencies of response to specific statements, but rather on categorisations derived from 
the interpretation of open-ended comments, several of which revealed misunderstanding of the 
question. Having said that, they reveal tentative (and not unanticipated) trends in supervisors’ 
emphases with regard to what students report that they receive. 

The feedback students report finding the most helpful 

Interviews with a subset of participants (n = 22) provided the opportunity to probe the types of 
written feedback that students reported finding the most helpful, and to consider their responses 
with reference to linguistic background (i.e., English as LI or L2). The next section summarises 
the main findings from the interviews, and illustrates the findings with specific examples. 

Students with English as LI 

For LI students, several issues of importance emerged from the interview data. Key issues were: 

1. LI students wished to receive direct (i.e., specific) feedback that would help them with the 
overall organisation of their writing. 

2. LI students also welcomed less direct comments that challenged their thinking and prompted 
them to find their own answers. 

3. Direct feedback on language was appreciated, but was seen as secondary to feedback that 
challenged the students intellectually. 

4. The quality of the relationship, and opportunities to discuss feedback as part of that 
relationship, were seen as important. 


Illustrative examples 

Student A presented herself as a highly motivated student who wanted “to get a high mark, so I 
want her to be critical with the content. Is it good? Is it worthy of a high mark?” In this connection 
she asserted “I do want her to give as much feedback as she can, really.” The type of feedback she 
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received was in the form of specific written comments ("change this", "move your references 
there") which, in her view, were focused on “all the big things rather than little details,” including 
“what I should cut, what I should expand upon, areas where I need to find more research.” These 
types of feedback were important to her “because I wasn’t really sure about how to write a 
dissertation.” For this student, comments that focused on content and organisation were more 
valuable than those that looked at the specifics of the language. This was because “the language I 
can eventually figure out for myself.” The type of feedback she most valued was on “what 
structure, what should go where, what’s necessary or not necessary, relevant or irrelevant.” 
Follow-up meetings provided the opportunity for clarification. 

Despite seeking this level of guidance, it was ultimately important for Student A that the work was 
seen as her own. She noted with regard to the written feedback, “she’s not trying to change my 
words, it’s keeping my own words.” Rather, the feedback was “pushing me in a direction so I can 
improve my own words. I think I'm given a lot of independence about what I should be writing.” 
Indeed, Student A concluded, “I don’t want her to take control, take ownership of it or anything 
like that, [but rather] give me some freedom, show me what I should or shouldn’t be doing.” 

It may be suggested that Student A represents a typical perspective of LI students who do not 
require specific feedback on their language, and for whom comments on content and organisation 
are more crucial. Indeed, Student A argued with regard to feedback on language that “I wouldn’t 
expect her to have to do stuff like that.” In contrast, Student B noted that, for her, direct feedback 
about the language was important: 

I think that I sometimes used colloquial language, and I hadn’t been aware of it, or 
there have been grammatical rules that I haven’t been aware of that I think right 
from the beginning [ he] has picked me up on, you know, like using a split infinitive, 
and things that I’m completely oblivious to. 

Student B went on to argue that “that’s really helpful because then hopefully I don’t keep making 
the same mistake.” Indeed, Student B noted that “my main supervisor goes through word by word, 
correcting my spelling and grammar,” with the consequence that “I think my writing has got better 
with each chapter.” For her, direct focus on the language was a very important dimension of 
feedback, regardless of language background. She observed, “one of my friends, her supervisors 
had quite a different approach, and so she got to the end thinking that it was okay, and she 
suddenly had to go back to correct things.” 

At a deeper level. Student B also reported that she received helpful feedback on organisational 
matters. Here, the comments were not necessarily detailed. Rather, they operated as prompts for 
the supervisor about issues that would be raised in subsequent discussion. Student B gave the 
example of “feedback about ... signposting, you know, letting the reader know what was coming 
up.” She commented, “I don’t think he wrote the word ‘signposting’,” but in the subsequent 
discussion “when he got to it ... he would have seen this note to himself, and [then] pointed out 
what I needed to do, or I think he asked me what was I trying to say here.” 

These two examples (Student A and Student B) reveal somewhat different priorities with regard to 
what was considered important feedback, although both regarded feedback that extended their 
thinking, and that could be discussed in follow-up meetings, as important. Furthermore, like 
Student A, Student B considered that the ownership still resided with her about what to do with the 
feedback. That is, “neither of them [my supervisors] said, ‘Oh, look, you should do it this way.’ 
That was left up to me. I think [my second supervisor] had some suggestions, but I think we just 
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all felt, let’s see what I come up with Student B conceded that “in terms of the grammar and 
editorial changes, yes, I could [get by without a meeting].” Nevertheless, “when it’s more about 
the topic and the content, then I would still want to talk about it.” 

The case of Student C provided a further example of the issues raised by A and B. Noting that she 
valued feedback because, in her view, “with no feedback on writing comes no real engagement by 
the supervisor in the project,” she explained that feedback operated at several levels (direct 
comments, comments that challenged her thinking and comments that invited subsequent 
discussion), and that she was willing to take the comments seriously, although ultimately the work 
was her own: “He [my main supervisor] said to me, ‘these are just my opinion, you know, you can 
take or leave every comment I make here.’ 99% of the time I used what he suggested.” 

Student C reported that at some points the feedback would be quite directive: “things like, ‘well, 
maybe you should reword this sentence’.” At other points it operated similarly to Student B’s sign¬ 
posting example - “he’d say ‘let’s discuss’ at certain points.” This balance of feedback (open- 
ended "let’s discuss" alongside more direct comments on which “Ed think, ‘oh yeah, I can do 
that’”) was helpful for Student C, even though it left room for a small element of doubt about what 
to do with the feedback: 

He would write down, like he’d say ‘you should expand on this notion’, and then I’d 
think maybe I should just check with him what he means by that, but I could quite 
happily work through all of this, I could read it. ... For me it worked to have 
something like this where I could sit down in my own time and think about what it is 
that he’s saying. 

In Student C’s thinking, opportunities to discuss the feedback in a positive way were sometimes 
just as crucial as private reflection so that she could understand and implement it: 

It was great, you know, you felt like he was really engaged with what I was doing, 
and really taking an interest, so that if I didn’t actually spend that much time with 
him I knew that I could just pop in ... and he’d always be able to be constructive. 

For Student C, a key factor in making the supervisor-student relationship successful was a 
sense of equality in the relationship. When they met “it felt like we were both on an equal 
footing, we were both having a discussion like normal human beings about a topic that we 
both had a similar understanding of.” She mused, “I can go and have a coffee with [him] 
and we can talk about stuff, and that’s like a normal conversation rather than teaching me 
something every time I see him.” 

Students with English as L2 

It may be hypothesised that students for whom English is L2 may hold different 
perspectives on the type of feedback and relational dynamic they find most helpful and the 
ways in which they understand and act on feedback. Several important issues for L2 
students emerged from the data, which revealed similarities and differences with the 
perceptions of LI students of what was effective: 


1. There was evidence of appreciation for direct feedback on language. 


9 



Journal of University Teaching & Learning Practice, Vol. 9 [2012], Iss. 2, Art. 7 


2. Balanced feedback that incorporated direct feedback on both language and organisational 
matters, alongside more indirect prompting, was perceived as important. 

3. In some cases L2 students were more apprehensive about interpreting the feedback than 
their LI counterparts, and wanted to be more carefully directed. 

4. However, promoting autonomy and developing intellectual capability were identified 
characteristics of effective feedback, as was the quality of the relationship. 


Illustrative examples 

Student D noted that “one thing that I really appreciate is that he looks at the grammar and helps 
me with these things.” Student E also noted that it was “useful, too” when “they correct the 
language errors.” Indeed, for Student E, pointing out specific language errors is “easier than 
getting overall feedback when you have so many mistakes [and] you will have to go find the 
errors.” Direct feedback on language meant that “you know exactly what to change to make it 
better. Sometimes 1 just cannot notice errors, especially sometimes use of plurals.” Even when “I 
get my friend to proofread and edit it before I give [it to] my supervisor” there were still errors that 
needed to be corrected. 

Student D and Student E also valued feedback that pushed them to think about their ideas and 
organisation, and the opportunity to discuss this feedback in follow-up meetings. Student D noted, 
“he writes some comments and gives [them] back to me, and then we have a meeting after that, 
and then we discuss the ideas.” These ideas might include “more literature review or some authors 
that he thinks I should read, or which ways I can develop my thesis.” The feedback was given by 
the use of specific questions asking Student D to think about issues ("What do you mean by ...?" 
"I’m not sure that I follow ..."). After this written feedback came what Student D described as the 
“crucial” follow-up meeting. This meeting provided Student D with the important opportunity not 
only to clarify comments but also to “defend my ideas” so that she ultimately took ownership of 
the work. 

The crucial importance of follow-up meetings was brought out by Student E, who noted that 
occasionally it was hard to understand the feedback, and “sometimes they ask questions, like ‘do 
you mean ...?”’ She reflected, “does she mean I misunderstood something or do I need to say 
more? [I am] sometimes not certain what I need to do.” She reiterated this later on in her 
interview: “the questions sometimes confuse me, do I need to change completely? Or am I just 
needing to add something? It’s not very clear to me sometimes what to do.” Feedback, both 
written and oral, was therefore needed that “indicates clearly what I need to do.” This level of 
uncertainty about interpretation also influenced Student F who noted that “I prefer specific errors 
[to be pointed out].” In her view, the most helpful feedback was specific and directive, “when she 
writes out in detail what exactly she wants me to do, or what she expects me to come up with.” 
This feedback was then discussed in follow-up meetings. 

Student G brought out a contrasting perspective that suggested that direct feedback on language 
was secondary to more indirect feedback on content and organisation. She reflected that, for her, 
feedback was “constructive” when it was “all about how they [my supervisors] help me to develop 
my ideas in writing” in a way that “encourages me to think more about my writing.” For Student 
G, grammatical accuracy was not the priority for feedback, “because we can do that later.” 
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The most helpful feedback for Student G was indirect in nature. That is, “[m]y supervisors use a 
lot of ‘you may want to...’ [or] ‘would you like to?’ ... they are so supportive, they just use 
indirect questions all the time.” Reflecting the perspective noted by Student C (an LI English 
speaker). Student G observed that this type of feedback was “absolutely helpful for me because ... 
I like to think, spend a lot of time thinking about my own writing.” Posing specific questions 
“makes me want to develop my own writing.” 

Once more reflecting Student C’s perspective. Student G also noted that follow-up meetings 
(whether formal or informal) with a supervisor who was on an equal footing were necessary parts 
of the process: 

It’s not just written feedback ... we talk about our draft over coffee, which is more 
relaxing ... it’s not just ‘I give you the draft and you give me feedback’ and that’s it. 

The most important thing is [to] talk about it. It doesn’t have to be a formal meeting, 
it can be, you know, you see each other in the hall, and you just come up with ideas 
or articles to read. Being friendly and [having] communication is very important 
between supervisors and student, that you keep talking about the work, and other 
things like life and ‘what did you have for lunch?’ - that kind of thing. 

Student G made it clear that ultimately it was up to her to process the feedback and decide what to 
do with it: 

They give me time to read my own draft, and it’s up to me what I need to put in my 
writing. They are not telling me what to add, or what not to do... They just put a 
mark here, ‘you need to think about this part; would you like to delete the whole of 
this, and move a bit of this one up to here?’ They’re just suggesting. 

Student H’s experience was that balanced feedback that (1) focused on the language as well as the 
content and organisation, and (2) was both direct and indirect, was the most beneficial. That is, her 
supervisor “would delete what I wrote, and then put a comment here ... she would also correct my 
grammar as well, because English is not my mother tongue.” She noted, however, that often 
feedback would be indirect, with a view to provoking the student to find her own answers: 

Mainly the literature review, she asked me to rewrite my literature review because 
she doesn’t think I’m specific enough, I did not provide examples to back up my 
claims, so she would ask me to rewrite, which I did. ... [But] she doesn’t tell me 
what to change ... she doesn’t give me straight answers, I was asked to find the 
answers myself. 

This more indirect prodding and questioning style had its advantages because: 

I learn things that way, about research ... with the content and structure, I learn 
something from them, and I will not make the same mistakes on this next time ... 
it’s hard for me to change my grammar competence immediately, it’s over a period 
of time, whereas the knowledge on how to do research projects is useful for my 
future studies. 

Nevertheless, for Student H, feedback on the language was also important because “[h]aving 
grammatical accuracy, it shows that I treat this report seriously, no mistakes.” 
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Taking evidence from all the interviews into account, it appears that being an LI or L2 speaker of 
English (and, by implication, bringing different cultural assumptions into the supervisor-student 
relationship) was not a differentiating feature of the type of written feedback students reported that 
they received and found helpful. Differences in language/cultural background also did not appear 
to contribute substantially to differential understandings of how the supervisor-student relationship 
should operate. Although it seems that L2 students relied more on feedback on language than their 
LI counterparts, there was some evidence to suggest that LI students might also benefit from this 
kind of direct feedback. Regardless of language and cultural background, our findings suggest that 
a range of feedback, both direct and indirect, is likely to be more beneficial and welcome to 
students than only one type of feedback. Thus, several categories identified, for example, by 
Gulfidan (2009) were in evidence. 

It was also perceived as important to receive feedback in both written and spoken form, that is, in 
the context of ongoing "learning conversations" (Wisker et al. 2003), where input was used for the 
purposes of feedback and feedforward (characteristics of effective feedback identified by Hattie 
and Timperley (2007)). The evidence suggests that students did in fact receive regular face-to-face 
meetings, both formal and informal, subsequent to receiving written feedback, and that these 
provided opportunities to discuss “Where am I going? How am I going? Where to next?” (Hattie 
& Timperley 2007, p88). 

It was apparent that the supervisor must also operate as a "mentor" and "mediator of 
knowledge and experience", as well as a "dialogue partner" (Franke & Arvidsson 2011). To 
fulfil these roles, the use of more indirect feedback that, in Student H’s words, helps the 
student to “find the answers myself’ was perceived as particularly effective, and seemed to 
be appreciated by several students in this study. Thus, although Gulfidan (2009) observed 
that students appeared to prefer clear, straightforward and specific feedback, the students in 
this study did not object to more indirect comments - provided they had the opportunity to 
discuss these in a supportive relationship. 

Advice for new supervisors 

As stated earlier in this article, participants were asked in the questionnaires to provide two 
suggestions about the most helpful type of feedback new supervisors could give to students new to 
the process of postgraduate research. This question provided the opportunity for participants to 
reflect on the feedback they were currently receiving, and, as a consequence, the issues they 
considered to be the most important in terms of providing feedback, in light of their own 
experiences. Interviews provided participants with the opportunity to expand on and explain their 
suggestions. Table 3 provides a rank order of suggestions made in the questionnaire. 

TABLE 3: Advice that Postgraduate Students Would Offer to New Supervisors 


Advice 

Total/53 

Percentage 
of responses 

1. Give written and oral feedback with a view to feed forward 

12 

23% 

2. Make positive and constructive comments alongside critique 

11 

21% 

3. Understand the project 

5 

9% 

4. Give suggestions but do not be too directive 

3 

6% 

5. Other/no suggestion 

22 

42% 
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Three out of four participants had reported that they already received written feedback in 
conjunction with follow-up meetings (see Table 2 above). The most frequently cited suggestion 
for new supervisors (Table 3) reiterated the perceived importance of this practice. Also, around 
one in five participants noted that it was important for written feedback to be positive and 
constructive as well as providing critique. 

Interview comments corroborated and elaborated on findings from the questionnaires. For 
example, reference was made to the importance of providing written and spoken feedback. The 
two must work together, otherwise a lot of feedback makes no sense (Student B; Student F; 
Student K). Reference was also made to the need for supervisors to show interest in the topic 
(Student J) and to “use positive comments as well as ... things to change” because “it’s motivating 
to receive positive feedback” (Student E). 

Several comments also interpreted "effective" in terms of the relationship in which feedback was 
being given: 

• Frame the relationship as a partnership of equals from which both parties might benefit: 

• “I had one co-supervisor ... [who] treated me like she was my boss ... it doesn’t work for 
me” (Student D); 

• supervisors “don’t have to know everything, you know, be open to learn together or to 
create something together” (Student D); 

• the supervisor should be “a giver, a guide” (Student I). 

• Demonstrate genuine interest in the project: 

• be “more hands-on, engaged with the thesis and the research” (Student D); 

• show enthusiasm for the work (Student F); 

• show a willingness to read and give comments (Student I). 

• Recognise that ultimately the work is the student’s own: 

• “I think by this level students should be taking some control” (Student A); 

• “I really want comments ... [that] encourage me to think about how to develop the 
writing” (Student G); 

• “I think always the final decision should be made by the student, so they can be more 
responsible with the decisions they make” (Student L). 

It was evident, therefore, that students saw the process of developing their knowledge through 
feedback as part of the process of developing a relationship (Franke & Arvidsson 2011). 
Regardless of linguistic/cultural background, they wanted to see the supervisor/student association 
constructed in terms of a "partnership of equals" rather than, for example, as "manager/employee". 
They wanted supervisors to demonstrate genuine interest in their work. At the same time, students 
recognised that the work was their responsibility, and that ultimately they needed to operate 
independently (Grant 2005; Riordan & Loacker 2009). 

It was apparent that the limitations of what Carless et al. (2011, p396) describe as a “one-way 
transmissive view of feedback, with tutors simply making comments on completed student 
assignments” can be overcome by regular supervisor-student meetings, and lends itself to “the 
development of iterative dialogic feedback cycles” and “an interactive exchange in which 
interpretations are shared, meanings negotiated and expectations clarified” (pp396-397). It seems 
that where this can be done in ways that put the supervisor and student on equal terms and operate 
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as what Student C described as "a discussion like normal human beings", these can be very 
effective. 

Limitations and recommendations for further research 

Although we believe that the present study has laid the groundwork for further research in the area 
of postgraduate feedback practices, it was framed as an exploratory study, and it is important to 
acknowledge several limitations. 

One of the aims of this study was to investigate similarities and differences among students who 
spoke English as LI or L2. Although language background was used as a differentiating factor 
when examining interview comments, cultural background was not taken into account (for 
example, we did not differentiate between Confucian Heritage Culture students and students from 
other backgrounds). This constrains the amount of comparison that can be made between different 
cultural groups. Also, a number of zero responses were recorded in questionnaires. This could 
indicate that some students had given little or no thought to the issues raised in the questionnaire. 
Where follow-up interviews did not occur, there was no opportunity to revisit the questionnaire 
and to seek clarification. 

Further research might include a more closed-ended questionnaire containing a series of 
behavioural and attitudinal statements, and both Likert-scale and ranking questions, which might 
help to mitigate nil responses. Certainly, the data gathered from this study could form the basis of 
developing the statements. Also, further studies should aim to include a larger sample size, and 
clear differentiation among languages and cultural backgrounds. These would provide a more 
solid basis for investigating differences across cultures. 

Additionally, this study did not specifically target students at different stages of the 
research/supervision process (although one criterion for inclusion was that this should represent 
the student’s first research degree). Further work might include a comparative study into the nature 
of feedback given at early and later stages of the supervision process. Also, a longitudinal study 
might help to gain insights into effective practice over the entire supervisory period, from the 
perspectives of both supervisor and supervisee. Such a study might investigate practices in the 
light of what may have been established in any supervisor/student agreements that were entered 
into at the beginning of candidature. 

Finally, we found no evidence from the interviews of L2 students’ deference to the "authority" of 
the supervisor, or requests for feedback that were at odds with the “problem solving dialogue with 
experts mode of European and western research” (Wisker et al. 2003, p384). On the contrary, 
several L2 students appeared to appreciate a more equal supervisory relationship alongside open- 
ended feedback comments that challenged them to think. It may be, although this is purely 
speculative, that the F2 students we interviewed were sufficiently inculturated into a New 
Zealand/Western approach to research study that they had begun to adopt a more independent, 
"problem-solving" stance. It may also be that these students, in deferring to us as researchers, gave 
us the responses they thought we were looking for. Further research in this area is therefore 
needed. 
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Conclusions 

Pearson and Kayrooz (2004) argue that effective supervision is critical to students’ successful and 
timely degree completion. Alongside Wisker et al. (2003), they acknowledge an expansion of the 
numbers of postgraduate research students, both local and international. With these factors in 
mind, the study reported here, although exploratory in nature, has sought to identify what students 
across a range of disciplines, and for whom English is LI or L2, say is working for them in terms 
of feedback on their work as part of an ongoing "symbiotic" supervisory relationship (Li & Seale 
2007). Our findings reveal a somewhat unified student perspective on the supervisor-student 
relationship that suggests that supervisors should give careful thought to the nature and dynamics 
of the relationship. The findings also reveal a feedback scenario that, at least for the students in 
this study, is obviously working. The students report clear examples of practices that, from their 
perspective, are effective. 

As an exploratory study providing preliminary findings on what appears to work in terms of 
written feedback in the context of "learning conversations", the study has successfully identified 
instances of current "good practice" alongside areas for development. Our findings also concur 
with those emerging from the supervisor questionnaire and interview data we collected, the 
purpose of which was to provide a means of comparing and contrasting how supervisors and 
students, as the two partners in the learning conversations, constructed effective supervisory 
feedback practices (Bitchener, Basturkmen & East 2010). These data indicated that, although 
supervisors held a range of beliefs about effective feedback, there was little difference in the types 
of feedback provided in the different discipline areas, and supervisors tended to give similar 
feedback to both LI and L2 students. 

Given the assertion that feedback is central to the development of students’ learning (e.g., Carless 
et al. 2011; Hattie & Timperley 2007), and the acknowledgment (Pearson & Kayrooz 2004; 
Wisker et al. 2003) that the postgraduate research student cohort is growing and diversifying, we 
suggest that the lessons learnt from this study, although relatively tentative, do contribute towards 
greater clarification of what students from a range of backgrounds (both disciplinary and 
linguistic) say works for them in the supervisory process. 
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